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ABSTRACT 


The purpose of this thesis was to design, conduct, analyze and report the results 
of a C3 experiment. ‘rhe T4 Simulation, acting as a surrogate for a C3 system, was used 
to generate data for statistical analysis. The objective was to determine which factors, 
and which factor levels, effected the MOE. As a result, the optimal system alignment 
was determined which would result in maximum values for the MOE. 

The factors investigated were Area, Communication, and Tactical Delay, and 
probability of winning a same turn conflict, P(W). The levels of delay varied from zero 
to nine moves while levels for P(W) varied from zero to one in increments of one tenth. 

Analysis showed that only Tactical Delay effected the MOE and that only two 
levels, zero and one, of Tactical Delay significantly changed the MOE. Analysis also 
showed that the player with the higher value of P(W), regardless of Tactical Delay, 
achieves a positive MOE. Therefore, the optimal system alignment, under the 


constraints of the experimental design, would be to assign P(W) = 1.0 to one side, while 


assigning the maximum value of delay to the other. Thus, this game configuration 


Accesion For 


would maximize the MOE. 
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I. INTRODUCTION 


A. BACKGROUND 

The joint Command, Control and Communications (C3) curriculum is designed to 
provide a comprehensive operational and technical understanding of the field of C3 
systems as applied to military operations. Students are introduced to modeling, systems 
architecture and engineering, and evaluation of C3 systems along with a basic knowledge 
of the physical principles and technologies that comprise communications systems, 
computers and sensors. The joint C3 curriculum includes instruction in communications 
and sensors, information science and tactical analysis/operations research that supports 


C3 systems development [Ref. 1: p. 66]. 


B. PURPOSE 
This thesis concentrates in the area of C3 systems evaluation. The Naval 
Postgraduate School (NPS) offers a course in C3 systems evaluation primarily intended 
for students in the C3 curriculum. 
The course is designed to be one of the curriculum’s capstone courses. (C3 
experiments related to the C3 evaluation process are conducted during the 
laboratory portion of the course. [Ref. 2:p 2] 


The C3 systems evaluation course uses experiments involving simulations and 


wargames to help evaluate systems effectiveness. Typically, we evaluate, compare or 


test hypothesis about C3 systems or modifications to them. C3 systems evaluation allows 














the student to learn through experience how to design, conduct, analyze and report the 


results of C3 experinints. This thesis will utilize a simple game for the purpose of 
planning and conducting an experiment including hypothesis development, design of 
expe:iment, conduct of trials, collection and analysis of data and reporting of the results. 
An enhanced version of the Tic-Tac-Toe (TTT) game called Tactical Tic-Tac-Toe 

(T4) was developed by Gary Porter to fulfill the need for the first experiment in the C3 
Systems Evaluation course. It is used because it: 

1. is easy to learn, 

2. is new to all subjects, 

3. is a non-military game, and 


4. generates data that are C3 related. 


T4 is a two-sided game. Players are assigned missions to achieve and their score 
is based on how well they achieve them. Players can play individually against an 
individual opponent or can be in teams of two playing against other teams of two. 
Missions are either to defeat or tie your opponent in the assigned game board (mission) 
area. 

Mission effectiveness depends on the degree to which you have achieved your 
assigned mission. A victory mission requires that you score more TTTs in the assigned 
mission areas than your opponent. While a survival mission requires you to at least tie 


your opponent in the assigned mission area in order to achieve your mission. Mission 





success is based on the total number of TTTs achieved in an assigned mission area 
relative to your opponent. 

When large amounts of experimental data are required, using human subjects to 
play T4 and to collect the data is time consuming and error prone. To alleviate these 
problems, Porter wrote the T4 Simulation which simulates human play, generates a 
wealth of data and collects the data for analysis. The data generated enable the user to 
Statistically study how different inputs into the T4 game, the surrogate for a C3 system, 


effect mission outcome. 


1. Research Questions 

The course in systems evaluation referred to earlier is a C3 course 
appropriately called C3 Systems Evaluation. The course requires students to participate 
in and lead an experiment that uses either wargames or simulations to generate data for 
analysis. It is in this course that students are exposed to the T4 simulation. The course 
is not long enough for students to pursue a lengthy experiment. Thus, many of the 
questions a student may want to attempt to answer may not be considered. It is for that 
reason this thesis is being written. 

This thesis will investigate how the T4 simulation responds to different types 
of intelligence delays. The delays will be investigated at several levels. The effect of 
changing the probability of winning a same turn conflict, P(W), will also be investigated. 
Finally, this thesis will investigate the effect delay and P(W) combined have on mission 
outcome. The following research questions will be answered using data generated as a 


result of multiple T4 replications: 




























As delay levels increase, mission margins of victory, MOVs, decline. Is there 
a level of delay above which further increases have no further effect on mission 
MOV? 


2a. Does changing the probability of winning a same turn conflict, P(W), effect 
MOV? 


2b. As the level of P(W) decreases, is a level reached beyond which P(W) has no 
effect on MOV? 


3. Does changing the probability of winning a same turn conflict, P(W), effect 
mission outcome regardless of Time Delay? 


4. Does an interaction exist between Time Delay and P(W)? 


These research questions require some discussion so the reader is clear on 
exactly what is to be accomplished in this thesis. Mission outcome refers to the number 
of TTTs that a team scores. Relative to the opponents TTTs, this difference in scoring 
is referred to as the MOV. As intelligence delays are increased for a particular side, it 
is expected that the number of TTTs scored by that side will decrease. However, it is 
possible that beyond a certain level of delay, the number of TTTs will not continue to 
decrease but will level off at some value. Delays are discussed in greater detail below. 

Conflict resolution is also described in detail below. So for now it will just 
be said that during T4 games, because of the simultaneous move rule, it is possible for 
two opposing players to attempt to enter the same game cell on the same move. Since 
only one player is allowed to occupy a cell, a conflict results. In the past, these conflicts 
were resolved randomly with both sides assigned the same probability of winning, P(W) 


= 0.5. The game now allows the probabilities to be weighted favoring a particular 


player. The thesis will evaluate the mission outcomes when one side is assigned a 
greater P(W) than the other side. 

The MOV for the different P(W)-Time Delay combinations may indicate that 
P(W) for example, has no effect on MOV at some levels of Time Delay but has a 
significant effect at others. These P(W)-Time Delay combinations that significantly 
effect MOV are called interactions and the data will be analyzed to see if any exist. 

The reader may not be able to adequately understand the posed questions until 
a discussion of T4 is presented. A thorough presentation of T4 is provided below. It 
begins by describing the simplest of the many T4 variants, the T4 Baseline game, and 
finishes with an explanation of the T4 Simulation. After the reader understands the T4 


game better, it is suggested that the above Research Questions subsection be reread. 


2. Approach 

T4 simulation was used to generate the data that was analyzed to answer the 
above research questions. The first question was addressed in a previous thesis by 
Jeffrey S. Richardson. Richardson hypothesized that as time delay levels increase, 
mission outcome would decrease up to a point beyond which further increases would 
have no effect on mission outcome. While Richardson’s data showed a significant 
relationship between Tactical Delay levels of zero and one, it failed to show that further 
increases, at some point, would have no effect on mission outcome. In fact, his data 
showed very little relationship between an increase in time delay and mission outcome. 
It is believed though, that there is a delay value above which mission outcomes no longer 


decline, but the number of replications of the T4 simulation used in Richardson’s thesis 











(30 replications per trial configuration) was insufficient to show this. Richardson’s 


experimental design will be used to answer question one using 120 replications instead 
of the 30 that he originally used. If levels of delay are found beyond which mission 
accomplishment ceases to decline, then those greater delays will be omitted from this 
thesis in order to more fully concentrate on the remaining research questions. 

The remaining research questions required a change in the T4 software to 
allow the user to input different levels of probability for same turn conflict resolution. 
Before the software change, when same turn conflict was resolved randomly, each side 
had the same probability of winning the conflict. To answer the second research 
question, this thesis investigated what would happen if one side had a higher probability 
of winning a same turn conflict. The data generated to answer Question 2 was used to 


answer the remaining research questions. 


C. TACTICAL TIC-TAC-TOE (T4) 


1. T4 Baseline 
Several variants of T4 exist but it is not necessary to discuss all of them in 
order to understand the game. In the next few sections, variants that will enable the 
reader to understand the version of the game used with the thesis will be discussed. The 
T4 Baseline game is the most fundamental of all the versions and will provide the reader 
with a good foundation to understand the T4 game variant used in this thesis, T4 


Simulation. The T4 Baseline game is very similar to the well known game of Tic-Tac- 








Toe (TTT) and it uses the same three cell by three cell game board. The differences will 


be explained below. 


a. Simultaneous Moves 
Both players announce their moves simultaneously and plot them on the 


game board. 


b. Conflict Resolution 
Because moves are made simultaneously, there is a good possibility that 
both players will chose the same cell. When this occurs, it is called a same turn conflict 
and the winner is chosen randomly, flipping a fair coin. The winner remains in the cell 
while the loser is not allowed an alternate move and simply loses that turn. A variant 
of this game alternates same turn conflict resoiution between players after the first 
conflict is decided randomly. 
c. Scoring 
Unlike the game of TIT, T4 does not end when the first TTT occurs. 
Instead, T4 is played until all nine cells are filled. When the game board is filled, the 
player with the most TITs wins and is awarded one point. Recall there are eight 
possible ways to make a TTT (three horizontal, three vertical, and two diagonal). It is 


possible for a player to get all eight TITs if every move results in a conflict and the 


same player wins every conflict. 














2.  TT4 With Intelligence Delay Factor 

This version is the same as the baseline game except for the addition of an 
enemy information delay factor. This factor, delays knowledge of opponents moves. 
The level of delay can be varied from zero to nine moves. The delay level is assigned 
before the game starts and can be set at the same or different levels for both players. 
When intelligence delay is introduced into the game, another type of conflict occurs and 
is termed a different turn conflict. 

A player that is assigned a one move intelligence delay, for example, will rot 
know which cell the opponent moved into until the following move. Therefore, it is 
possible the player assigned a delay may choose a cell that is already occupied. When 
this occurs, the conflict is always won by the player first occupying the cell. Like the 


baseline game, the loser is not allowed an alternate move and loses that turn. 


3. Double Game Board T4 
This version uses two TTT boards to form one larger board three cells high 
and six cells wide. Although it is considered only one board, the sides are designated 
as left and right game boards. In this game a turn consists of each player making a move 
on both the left and right game boards. These moves, like the baseline game are made 


in secret and announced simultaneously. Likewise, same and different turn conflicts are 


resolved as in the baseline game. 











a. Crossover Scores 


Unique to the double game board version are TTTs that cross from one 
game board side to the other. These are called crossover TTTs and there are ten possible 
ways for these to occur. Realize they can only occur horizontally or diagonally but not 
vertically. 

b. Intelligence Delays 

Like the T4 with Intelligence Delay game, the double game board version 
permits intelligence delays of zero through nine. Different delay levels are player 
independent. Different delay levels are also game board-side independent in that 
different delay levels may be assigned to different sides of the game board. All four 
players can be assigned different values of delay. For example, the X player on the left 
game board may have no intelligence delay while the right game board X player has an 


intelligence delay of several moves. 


c. Mission Assignments 
The assignment of missions in this version allows players to achieve an 
objective within a specific area of the game board. The objective assignments are either 
victory or survival. Victory is achieved by scoring more TTTs than an opponent in an 
assigned mission area while survival is achieved by either scoring a victory or tieing an 


Opponent in an assigned mission area. 











The specific areas of the game board that a player may be assigned an 


objective are left, right, crossover and overall. The following is a complete list of the 


mission areas a player may be assigned and a brief description of how each is achieved. 


1. 


2. 


3. 


4. 


a: 
6. 


VL = Victory left. By scoring the most TTTs on the left side. 

VR = Victory right. By scoring the most TTTs on the right side. 

VC = Victory crossover. By scoring the most TTTs in the crossover area. 
VO = Victory overall. By scoring the most total TTTs. 

SL = Survival left. By not losing the left side. 

SR = Survival right. By not losing the right side. 

SC = Survival crossover. By not losing the crossover area. 


SO = Survival overall. By not losing overall. 


Each player can be assigned up to four missions. Like intelligence 


delays, different players can be assigned different missions. It should be evident at this 


time that there are numerous game configurations to choose from in Double Game Board 


T4. 


Team T4 Games 


Team T4 is a natural progression of the double game board version and 


introduces additional players, a controller and three types of intelligence delay. 
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a. Players 

Team T4 is made up of two teams each with two players. The X team 
consists of an X Left (XL) player and an X Right (XR) player. Similarly, the O team 
is made up of an © Left (OL) player and an O Right (OR) player. Mission assignments 
are always the same for both players on the same team but opposing teams may be 
assigned different missions. This game is played like the other versions but tends to get 
complicated because of the increased number of delay factors which will be discussed 
below. To deal with the increased complexity of the game, a controller is assigned to 


monitor its progress. 


b. Controllers 

A controller administers the game, ensuring conflicts are resolved fairly 
and player’s game boards are adjusted correctly taking into account the various delay 
factors. Like the other T4 versions players choose their moves in secrecy but instead of 
announcing them to theiz opponent, they give them to the controller. The controller then 
updates each individual game board and returns it to the player. The controller also 
maintains a master game board that represents ground truth, the actual status of the game 
without any intelligence delays. The intelligence delays that have been introduced into 


this version will now be presented. 


c. Delay Factors 
Intelligence delay has been expanded into three categories and they are 


described below. 
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1. Tactical delay. This delay refers to knowledge of own enemy move information 
(the opponent on the same side of the double game board). 


2. Area delay. This delay refers to partner’s enemy move information. 


3. Communication delay. This delay refers to partner’s move information. 


These delays put constraints on the timeliness of receiving move 
information, by any one player, from the other three players. Each of the four players 
can be assigned different values for each of the three different delay types. These 


values, like the original intelligence delay, can vary from zero to nine moves. 


d. Summary 

At this point, the reader should feel reasonably comfortable that they 
have an understanding of the Team T4 game. To reinforce what has been discussed thus 
far, a quick review of the different factors that can be assigned to Team T4 games and 
the different levels some of these factors can take are summarized. 

Two players each are assigned to the X and the O team. The teams are 
divided into left and right side players. Each team is assigned up to four mission areas 
with an objective of either victory or survival in each area. Recall that both sides on the 
same team are given the same mission assignment while the opposing team may have an 
entirely different assignment. Recall also that three types of delays can be assigned to 
each player, each varying from zero to nine moves. These delays can be different for 


each of the four players. 
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Experience has shown that while game play is fairly simple, the 


administration of ine game is difficult, time consuming and susceptible to errors. 


5. Auto T4 
As might be expected, a computer assisted automation of the game was 
developed to assist the controller. This program is called Auto T4. 
This program allows pretrial configuration of the T4 games, computer assisted 
administration of game play and automatic scoring and collection of data. It is 
hoped that use of this aide by lead groups in 1991 T4 experiments will permit an 
increase in sample size and a reduction in data errors. [Ref. 3:p. 1] 


It is important to realize that other than what is described directly above, 


there is no difference between Auto and Team T4. 


6. T4 Simulation 

A natural progression from the Auto T4 game is the T4 Simulation program. 
It simulates four players participating in Team T4 games, complete with mission 
assignments and intelligence delay constraints. Since Auto T4 already contained the 
software to score the game, to collect all the data generated by the simulation and to 
record it in a data file, the T4 Simulation program was produced by the addition of two 
major changes. 

The first addition was a method to control the simulated play of each player. 
The problem with simulating human play was creating the software to do it. To 
accomplish this, each simulated player is controlled by a set of three game plan matrices. 


The second addition allowed multiple trial runs to be accomplished without 


user intervention between trials and between configurations. A game file, that contains 








all configuration information, allows the user to specify multiple trial configurations and 
the number of replications to be run for each configuration. 

The original purpose of the T4 simulation was to act as a surrogate for human 
play in a pilot experiment to discover a set of T4 variables of interest for later use in 
trials with human subjects. It was thought that this would be beneficial in several ways. 
Using simulated trials is extremely cost effective in terms of time required. 

Simulation permits the use of a much larger sample size resulting in a higher 
degree of resolution of results inferred from data. Use of simulated players thus 
adds greatly to the degree of control and independence of the trial outcomes. 
Humans make mistakes, learn from experience, play differently with different 
partners and other sometimes unknown factors that serve to add noise to the data 
in which you are trying to find a signal. [Ref. 4:p. 1] 

This chapter introduced the T4 game to the reader. The purpose of the thesis 
was discussed including the research questions and the approach taken to answer them. 


The following chapter will describe the experimental design that was used to generate the 


data. 
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Il. EXPERIMENTAL DESIGN 


A. SETUP 

In presenting the setup of an experimental design, the physical setup, the test 
subjects and any special equipment should be described. This is fairly simple when a 
simulation is used to generate the data. When using a simulation, all that is required is 
a computer and the necessary software. While using human subjects in the experiment, 
other issues such as assignment of teams and controllers, scheduling of when and where 
game play will be conducted, keeping score and data collection must be addressed. For 
completeness though, the hardware and software used to run the simulation and to collect 
and analyze the generated data will be described. 

1, Physical 

Gary Porter’s T4 Simulation was run on a Macintosh micrccomputer in 

Hypercard 2.0. T4 is written in Hypercard. It contains the code which simulates game 
play and then sends the generated data to a data file. The data is copied to disc in text 
format and then imported into Minitab or Microsoft Excel, the two software packages 
used to analyze and graph the data. Minitab was used as a Statistical tool to do the 
majority of the analysis while Microsoft Excel was used to manipulate the data and 


produce the graphs shown in chapter three. 
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2. T4 Simulation 


Before discussing the simulation setup, it is important to realize that this 
thesis does not fully utilize the full potential of the T4 Simulation in generating different 
configurations. As the simulation setup is described, it will be pointed out where the full 
range of possible parameters was not used. 

The setup of the T4 Simulation requires the assignment of the following 
parameters. They need not be assigned in the order listed below, but they all must be 
assigned before the simulation can run. 


1. player intelligence delay levels (Area, Communication and Tactical) from zero 
to nine turns, 


2. three player strategy matrices which control simulated player game play, 
3. team missions (up to four survival or victory mission areas), 
4. the number of replications per configuration, and 


5. methods for resolving same turn conflicts. 


The first research question attempts to answer a question asked by Richardson 
in his thesis. Therefore, his T4 Simulation setup, except for increased sample size, will 
be used to generate data to answer Question 1 and is presented below. To answer 
research questions two through four, some changes to Richardson’s setup were required. 


Changes to his setup are clearly identified as they are discussed. 
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a. Intelligence Delays 

Team X was chosen as the control team, the team with zero information 
delay throughout the experiment. The O team then, received information delays during 
the experiment and as such, was considered the treatment team. During the experiment, 
both simulated O team players, left and right side, always received the same level of 
information delay. Each of the three types of delay were investigated separately. For 
example, when Area Delay was increased from zero to nine, the delay levels for 
Communication and Tactical Delay were held constant at zero. All nine levels of delay 
were investigated. 

When data was generated to answer questions two through four, only 
Tactical Delay was considered, the other two types of delay (Area and Communications) 
were not. For Tactical Delay, only levels zero through three were investigated. The 
reasons for this will be discussed in the next chapter. Therefore, the full range of 


available parameters was not utilized while generating data for these questions. 


b. Simulated Player Strategies 
After determining the control and treatment ieam, strategies were 
assigned to control the simulated play of each player. Discussed briefly in the first 
chapter, strategy is controlled by a set of three game plan matrices. These game plan 
matrices include: 


1. The Cell Game Plan matrix which determines where a player will move at the 
start of a game. It also assists in resolving same turn conflicts. 


2. The Regular Game Plan matrix which controls a players moves on the left and 
right game boards. 
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The Crossover Game Plan matrix which controls a players move in the 
crossover area of the game board. 


The game plan matrices are assigned point values by the user and the 


simulation uses these values to determine how the simulated players move. The way the 


players move represents their strategy. For an in depth discussion on how game plan 


matrices control simulated players, see Ref. 4. 


Representative categories of available strategies, based on the values 


assigned to the game plan matrices, that can be assigned to each individual player are 


listed below: 


1. 





Qverall Offense: Ensures the simulated player will attempt to maximize own 
TTTs while making no attempt to block opponents TTTs. 


Balanced Offense: Ensures the simulated player will primarily attempt to score 
own TTTs, but will also try to block opponent’s. If there is simultaneously a 
TTT possible for the player and the opponent, a tie, the player will make an 
offensive TTT instead of blocking the opponent’s TTT. 


Balanced: This is the same as Balanced Offense except in the case of a tie. 
When a tie occurs, the choice of either making a TIT or blocking one is 
random. 


Balanced Defense: Ensures the simulated player will primarily attempt to block 
opponent’s TTTs, but will also try to score their own. If a tie occurs, the 
player will block the opponent’s TTT rather than make one. 


Total Defense: Ensures the simulated player will attempt to block opponent’s 
TTTs while making no effort to score their own TTTs. 


Random: This strategy provides the player with no strategy, all moves are 
random. 





The strategies that were assigned for this experiment were offensive 
crossover, OC, for both left side players and defensive crossover, DC, for both right side 
players. OC, a variant of the Balanced Offense strategy, encourages the left side players 
to score TTTs in both the left and crossover areas of the double game board. DC, a 
variant of the Balanced Defense strategy, encourages the right side players to block their 
opponent’s TTTs at the expense of scoring their own in the right and crossover areas. 
These simulated player strategies are realized as a result of assigning different values to 


the game plan matrices. It should be noted that each player can be assigned one strategy. 


c. Mission Assignments 
The mission assignments were VL and VC for both teams. Because both 
teams were assigned identical missions the possibility of same turn conflicts was 
maximized. It was expected that this setup would be best for determining what happens 
when changing the P(W) is investigated. Recall the full range of possible mission 
assignments includes eight different missions, see p.9. The simulation allows each team 
to be assigned up to four missions. Once again the full range of available parameters 


was not utilized. 


d. Replications 
The number of replications per configuration was increased from 30 to 
120 to generate data to answer all of the research questions. It was believed that a 


sample size this large would result in a higher degree of resolution of the results inferred 
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from the data. The only limit to the number of replications is the number of rows 


available on the spreadsheet used to analyze the data 


e. Same Turn Conflict Resolution 

Finally, same tum conflict resolution was random for all conflicts, initial 
and subsequent. To answer the research questions two through four, the O team was 
assigned a different P(W) for each game configuration ranging from 1.0 to 0.0 in steps 
of 0.1. For example, the O team for the first 120 replications was assigned P(W) = 1.0. 
The X team then, was automatically assigned the complementary value of P(W) = 0.0. 
This probability applied to both initial and subsequent same turn conflicts. The T4 
Simulation program user can assign a P(W) from 1.0 to 0.0 for the first conflict that 
occurs. All subsequent conflicts can be resolved either randomly or alternately. 
Different P(W)s may be assigned for initial and subsequent same turn conflicts. 

The purpose for pointing out the full range of T4 configuration options 
is not to critique the thesis but to point out that many more configurations are available 


for additional study. 


B. PROCEDURE 


After the generated data was transferred to disc it was reduced and manipulated 


in Microsoft Excel and put into columns for analysis. Both the statistical package in 
Excel and Minitab require data to be in columns for subsequent analysis. The raw data 


extracted from the original spreadsheet consisted of eight columns. Four each for the X 








and O teams. Each four column group was labeled Scoring by TTTs and the individual 
columns were labeled as follows: 


1. TL which represents the total TTTs achieved on the left side game board. 


2. TC which represents the total TTTs achieved on the crossover area of the game 
board. 





3. 7R which represents the total TITs achieved on the right side game board. 


4. TO which represents the total TTTs achieved on the entire game board is the 
sum of the first three columns. 


Since the assigned missions did not include VR, the 7R values were eliminated and 
TO was recalculated using only the 7L and TC columns. The margin of victory, MOV, 
was then established using the following equation: MOV = TO, - TO,. In other words, 
the MOV is the difference between the total X team TTTs and the total O team TTTs 
in the assigned mission areas. It is the MOV columns that are statistically analyzed in 
this thesis and no further data manipulation was required. 

The following chapter discusses the statistical methods used to analyze the MOV 


and presents the results of the analysis. 
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II. ANALYSIS 


A. ANALYSIS PLAN 

The plan used to generate and analyze the data that answered the first research 
question was produced by Richardson for his thesis. Recall that the Measure of 
Effectiveness (MOE), referred to earlier as the MOV, he selected was the difference in 
total TTTs between the X and O sides in the assigned mission areas. The mission areas 
selected for both teams were VL and VC. Conflict resolution for same turn conflicts was 
random, P(W) = 0.5, for both initial and subsequent conflicts. The player strategies 
assigned as diescribed in chapter two were OC for both left side players and DC for both 
right side players. Recall also that Area, Communication and Tactical delays between 
zero and nine turns were applied to the O side only. Thirty replications were run for 
each configuration totaling 10*3*30 = 900 trials. 

After question one was answered, it was determined that Area and Communication 
Delays at any level and delay levels above one for Tactical Delay were not statistically 
significant. Ther<iore they were not considered while generating data to answer the final 
two questions. Unlike the plan that was used to answer the first research question, same 
turn conflicts were no longer resolved randomly with P(W) = 0.5. A change in the T4 
software enables the user to select the P(W) for the O team on each side of the game 


board. Four inputs are available. The user can assign a P(W) for the initial same tum 


conflict for the left side O player and a P(W) for subsequent same turn conflicts for the 





same player. The same options are available for the right side O player. The T4 
software assigns the complementary P(W) to the X player automatically. In this analysis, 
P(W) was set the same for each player on a team for both initial and subsequent same 
turn conflicts. P(W) ranged from 1.0 to 0.0 in increments of 0.1 for a total of eleven 
categories. The T4 simulation was run at these assigned P(W)s for each delay level (0, 
1, 2 and 3) for 120 replications per configuration resulting in a total of 11*4*120 = 


5280 trials. 


B. METHODOLOGY 
This section will discuss the statistical methods used to analyze the generated data. 
Not all the methods below were used to answer each question. The methods used will 


be referred to in the results. 


1. Distribution 

The techniques used to analyze the data required an assumption if they were 
to be valid: The population from which the sample is drawn must closely resemble a 
normal distribution. The first requirement then was to prove that the data was at least 
approximately normal. 

According to the Central Limit Theorem (C.L.T.), given a random sample 
from a distribution with mean » and variance o’, if the sample size is sufficiently large, 
the sample mean has approximately a normal distribution. A rule of thumb for 


sufficiently large is a sample size greater than 30. 
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Rather than just assume normality based on the C.L.T., two simple graphical 
techniques were used to support this assumption. The first technique was the histogram. 
The histogram provides a pictorial representation of the frequency distribution. If the 
data is normal, the tops of the columns generated by the histo,.am can be connected with 
a smooth line and the resulting curve will look similar to the standard normal curve. The 
second technique was the normal probability plot, which was generated to supplement 
the histogram in checking for normality. A normal probability plot that is close to a 
Straight line suggests that the assumption of a normal distribution is plausible. When 
these techniques show that the data is normal, the following tools can be used for the 


data analysis. 


2. Single-Factor Analysis of Variance 

Analysis of variance, ANOVA, is a Statistical tool that can be used to study 
the relation between a dependent variable, average MOV in this case, and one or more 
independent variables. The independent variables are called factors and are represented 
by different types of delay and P(W) in this thesis. Single-Factor ANOVA investigates 
one factor at a time while the other factor is held constant to see if the factor being 
investigated has any effect on the dependent variable. The question of central interest 
using ANOVA is whether there are differences in the sample means (y,’s) associated with 
different combinations of delay and P(W). The sample mean being referred to is the 
mean MOV. ANOVA determines whether the average MOV for a particular level of 
delay, a, is statistically different than the average MOV for some other delay level, a,, 


while holding P(W) constant. ANOVA will investigate all possible pairs of delay, a = 
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a,, Ag = a), ... etc., and determine if the average MOV between different pairs of a@ are 
the same or different. To answer this question the following null and alternative 
hypotheses are tested. 

1. Hy: wy = pw, = °° = py, 


2.  H,: at least two of the y,’s are different. 


If H, is accepted, it can be concluded that delay has no effect on average 
MOV. If H, is accepted, then average MOV differs among different levels of delay. 

The method used for choosing between H, and H, is evaluation of the F 
statistic. In general, large values of F support H, while values near one support H,. 
Both Minitab and Microsoft Excel generate the F test statistic. To determine precisely 
which alternative should be chosen, the F test statistic is compared with the F critical 
value. The F critical value is generated by Excel but Minitab users must look it up in 
tables of F critical values which are available in any statistics text. If the F test statistic 
is less than the F critical value, H, is accepted. Conversely, if the test statistic is greater 
than the critical value, H, is accepted. It should be pointed out that the F critical value 
depends on the assigned value of the significance level a. As the value of a decreases, 
the F critical value increases. For example, if H, is just barely rejected at a = 0.05, it 
is possible that at a = 0.01 the F critical value would become larger than the F test 


Statistic and H, could be accepted. 











3. Two-Factor ANOVA 

Since single-factor ANOVA investigates the effect of one factor at a time 
while holding other factors constant, it has some limitations. To investigate the effects 
of both factors, delay and P(W) simultaneously, a stronger analysis tool is required. 
Two-factor ANOVA provides this capability along with some important advantages over 
single-factor ANOVA. 

Two-factor ANOVA is more efficient than its single-factor counterpart. 
Single-factor ANOVA provides no information about the factor being held constant. As 
a result, numerous single-factor runs had to be run in order to investigate all possible 
P(W) and delay combinations. Two-factor ANOVA allows all data to be stored in a 
matrix and analyzed in a single run. 

The single-factor study also provides less information than the two-factor 
study. Although numerous single-factor runs were generated, no information about any 
special joint effects between delay and P(W) were provided. These joint effects are 
called interactions and indicate that the effects of each factor should not be discussed 
separately. When interactions exist, it must be determined if they are important. 

The determination of whether interactions are important or unimportant is 
admittedly sometimes difficult. This decision is not a statistical decision and 
should be made by the subject area specialist (researcher). The advantage of 
unimportant (or no) interactions, namely, that one is then able to analyze the factor 
effects separately, is especially great when the study contains more than two 


factors.[{Ref. 5:p. 687] 


Like single-factor ANOVA, two-factor ANOVA also has null and alternate 


hypotheses. However, they are not as simple as single-factor hypotheses and a 





discussion of the parameters used in them is necessary before they are stated. In two- 
factor ANOVA, values known as interaction parameters and main effects are what is 
compared. Let: 


1. B= V/ Liiuis 





2 . Ki. 1/J Lyi; and 


3. pj = VIE; y,. 
Where I and J represent the different levels of factor A and B. Factor A is 
Tactical Delay and has I=4 levels, while factor B is P(W) and has J= 11 levels. The 
data matrix then is I columns by J rows with each cell containing 120 MOVs, the number 
of replications for each Tactical Delay-P(W) combination. Thus p is the true grand 
mean, y;, is the expected response averaged over all levels of one factor while the other 
factor is held constant, and the same is true for »;. Therefore, y, is the mean of a 
column of data representing a constant time delay over all values of P(W). 
Now define: 
l. a; = 4,,- w = the effect of factor A at level i, 
2. 6B; = 4; - « = the effect of factor B at level j, and 


3. Yg = Hy- (4 + a+ B= the interaction parameter at level ij. 


There are now three sets of hypotheses that will be considered: 
1. Hoss: ¥g = 9 for alli) versus Hy: at leastone y; # 0 


2. Hoa: a, = = a, = Oversus  H,,: at least one a, #0 
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3. Hy: 8: =~ = By, =O versus Hy: at least one 8, # 0 


The interaction hypothesis, Hs, is tested first to determine if interactions 
exist. This hypothesis addresses Question 4. If H.x,, is rejected and the interactions are 
determined to be important, straightforward interpretation of the results is not possible. 
However, if the interactions exist and are deemed unimportant, the two factors can be 
analyzed separately. 

4.  Tukey’s Procedure 

If H, is rejected using single-factor ANOVA, it is important to know which 
,'S are significantly different from one another. Determining which y,’s are different 
will show how the different factor levels effect mission outcome. To conduct this further 
analysis, a multiple comparison procedure will be used. There are a number of these 
procedures in statistics literature. The one that will be used in this thesis is called 
Tukey’s Procedure. This procedure produces a collection of simultaneous confidence 
intervals about the true values of all pairwise differences between every sample mean. 
If an interval does not contain zero it can be concluded that the two means being 
compared differ significantly at level a. 

5. Simple Linear Regression and Correlation 

The objective of regression analysis is to exploit the relationship between two 

(or more) variables so that we can gain information about one of them through knowing 


values of the other(s) [Ref. 6:p. 453]. Minitab uses the least squares estimate method 
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to generate a fitted regression line. The slope of the regression line provides insight into 

the relationship of the independent variable to the dependent variable, the average MOV. 
If the slope is zero, the line is horizontal and the average MOV is constant and does not 
depend on the independent variable, delay. 

The sample correlation coefficient measures how strong the association is 
between two variables in a sample. It is always between -1 and +1. A positive 
correlation occurs when the MOV tends to increase as delay increases. A negative 
correlation implies the opposite. If there is almost no association between the two 
variables, then the sample correlation coefficient will be near zero. 

The methods above were used to make the final conclusions about whether 
to accept or reject the null hypotheses. However, it is always wise to graph data prior 
to applying more sophisticated analysis techniques. Graphing data provides a preliminary 
idea of what to expect from the analysis. Graphs, when compared to the results of 
Statistical methods, provide a visual description of what the results mean. Finally, 
graphical representation of data provides confirmation of results that may not be 
understood otherwise and should always be produced. In the following section, graphs 


will be used in support of results wherever applicable. 


C. RESULTS 


1. Time Delay 


Research question one asked if there was a value of delay above which 


mission outcome for the O Team ceased to decline. It was expected that at an increased 














number of simulation replications, the data would smooth, clearly indicating the delay 


level above which mission accomplishment no longer declined. The results of single 
factor ANOVA will either prove or disprove this expectation, but before considering 
these results a look at the data graphically will provide a gut feeling for what to expect 


from the analysis. 


a. Graphical Analysis 

Figure 1 is a graph of the average MOV for each of the three types of 
delay plotted at a constant P(W) = 0.5. This graph shows by how much on average the 
X side beats the O side as intelligence delay applied to team O increases for each of the 
three types of delay. The graph indicates a clear increase in average MOV between delay 
levels zero and one for Tactical Delay, indicating an increase in mission outcome for the 
team. It is difficult to determine graphically though, if there is any significant change 
in the average MOV among Tactical Delays beyond one. The graphs of Area and 


Communication Delay do not indicate any significant relationships are present. 


6. Normality Checks 
To answer the first research question, single-factor ANOVA was used. 
Before conducting ANOVA, the data was checked for normality. The histograms 
generated for question one data produced columns similar to the standard normal curve 
and the normal probability plots generated fairly straight lines. An example of each is 
in Appendix A. Given the large sample size and the positive results of the graphical 


techniques, the assumption that the data was normal appears valid. 
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Figure 1. Average MOV vs. Time Delay 
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c. Single-Factor ANOVA 

Appendix B has the results of the single-factor ANOVA conducted on 
each set of data. In the appendix Groups equate to time delays applied to team O. 
Comparison of the F test statistic against the F critical value reveals the null hypotheses 
for Area and Communication delays were accepted indicating no statistical difference 
between average MOVs at any level of delay. Comparison of the F values for Tactical 
Delay indicates rejection of H,, thus requiring turther analysis of the data to determine 
at which level(s) of Tactical Delay average MOVs differed. 

Before discussing any further analysis, a more thorough discussion of 
some of the ANOVA table is presented. 

The Summary section of the output consists of five columns and ten rows. 
The first column, Groups, represents each level of delay, Tactical Delays zero through 
nine and the Count column indicates the number of replications of each level. The total 
number of X team TTTs minus O team TTTs per replication, summed over ail 
replications is in the Sum column while that same number divided by the count is stored 
in the Average column. This column is the average MOV. Finally, the Variance column 
contains the sample variance, s’, about the average. Along with the assumption that the 
data is normal, an additional underlying assumption made in order to use ANOVA is that 
the different levels of delay have the same population variance, 0”. Since o? is usually 
not known, s? is used. The results show that the differences between them are negligible. 
However, if the assumptions about normality or variance seem doubtful, a knowledgeable 


Statistician should be consulted. 
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The section labeled ANOVA contains the values that determine which 
hypothesis is accepted. The first value in this column represents the mean square for 
treatments, MSTr. As the average MOVs become more discrepant, this value increases 
indicating H, is probably true. If o* were available it could be compared directly to 
MSTr. If MSTr was considerably larger than o” , H, would be rejected. The second 
value in this column represents the mean square error, MSE. This value is an unbiased 
estimator of o” and is the weighted average of the sample variances in the above section. 
The F test statistic is the result of dividing MSTr by MSE. 

The next section in the ANOVA table to be discussed is the column 
labeled SS, sum of squares. The first value in this column is the treatment sum of 
squares, SSTr, the second value is the error sum of squares, SSE, and the third value is 
the sum of the first two and is appropriately named the total sum of squares, SST. The 
SST is a measure of the total variation in the data. The SSE measures the variation 
within the different levels of delay and is the unexplained part of the SST. SSTr 
measures the variation between the different delay levels, and is the explained part of the 
SST. Like the MS values, the ratio of the SS values, if each is divided by the correct 
degrees of freedom, df, can determine the F test statistic. 

d. Tukey’s Method 

Tukey’s method was used to generate a collection of confidence intervals 
about the true value of all pairwise differences between average MOVs for Tactical 
Delay, see Appendix C. It is up to the researcher to investigate the intervals and identify 


those that do not contain zero. In the appendix, row and column labels represent delay 
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levels. The results show that Tactical Delay of zero, when compared with every other 
level of delay, was significantly different than all other levels. For example, the interval 
comparing delay levels zero and one ranges from -2.839 to -0.095 and does not include 
zero. Tukey’s analysis indicates there is no significant difference in average MOV past 
a delay of one turn. Tukey’s method by itself seemed to answer research question one. 
However, the graph of average MOV vs Time Delay, Figure 1, appears to indicate large 
differences between average MOVs beyond one for Tactical Delay. A straighter, more 
horizontal curve was expected if there were no significant differences between the 
average MOVs at delay levels one through nine. To confirm Tukey’s results and to 
discount the visual analysis, a fitted regression line was plotted over the delay levels of 
interest to analyze the association between MOVs at these delay levels. The correlation 


coefficient, r, was also computed. 


e. Corroboration of Tukey’s Method 

Simple linear regression was used to generate a regression equation, see 
Appendix D, to analyze the average MOV beyond Tactical Delay of one. The equation 
that is generated represents a line that is very nearly horizontal, slope = 0.065, 
indicating the MOV does in fact remain fairly constant after a delay level of one. Along 
with the results of linear regression, a correlation coefficient was generated. The 
coefficient is close to zero, r = 0.04, indicating there is almost no association between 
Tactical Delay and average MOV. This corroborates the Tukey results, i.e., the only 


difference between average MOV is the difference that exists between Tactical Delay 


levels zero and one. 








2. Changing P(W) 


The response to question one, showed that only Tactical Delay effected 
average MOV. Further, using Tukey’s method it was further determined that significant 
differences in average MOV only occurred between Tactical Delay levels zero and one. 
It was expected that the results from Question 1 analysis would allow elimination of 
insignificant delay levels. Analysis showed that significant changes ceased to exist 
beyond a delay of one. The remainder of the research questions introduce an additional 
factor, P(W). There was uncertainty regarding whether including P(W) would change 
the results beyond a delay of one. To protect against this outcome Tactical Delay levels 
from zero to three were used for the remainder of the analysis. The second research 
question investigates the relationship between MOV and P(W) as P(W) varies from 1.0 


to 0.0 in increments of 0.1 while the levels of Tactical Delay vary from zero to three. 


a. Normality Checks 
This configuration was also analyzed using single-factor ANOVA. The 
data was checked for normality and the results of histograms and normal probability plots 
confirmed it. Single-factor ANOVA was run at each level of Tactical Delay to determine 


the effect of changing P(W)s on average MOV at constant levels of Tactical Delay. 


b. Graphical Analysis 
The graphical analysis is based on Figure 2 which shows that a steady 


increase in MOV for the X team occurs as the P(W) for the O team decreases. It also 
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Figure 2. Average MOV vs. P(W) 














appears that increasing delay level for team O results in a higher MOV for team X, 


although the differences appear slight and are probably insignificant. 


c. Single-Factor ANOVA 
Based on ANOVA the null hypothesis was rejected all four times 
indicating that P(W) significantly effects average MOV at all four levels of delay. To 
determine which different levels of P(W) effect the average MOV, Tukey’s method was 


used once again. 


d. Tukey’s Method 

Tukey’s method generated confidence intervals revealing how pairwise 
levels of P(W) effected average MOVs. The results are in Appendix F. These results 
indicate that the majority of the pairwise comparisons have a significantly different effect 
on average MOV. At Tactical Delay zero for example, just three pairs of P(W) have 
the same effect on average MOV at a = 0.05. The interval comparing P(W) = 0.6 and 
P(W) = 0.5 contains zero, indicating the average MOV does not differ significantly 
between those two P(W)s. It can be stated then that at Tactical Delay 0, no significant 
increase in average MOV would exist between P(W) = 0.5 and P(W) = 0.6. The same 


can be said for all intervals that do not contain zero. 


e. Linear Regression 
Simple linear regression was used once again to determine regression 


equations for the data at each level of delay. See Figure 3. The slopes of these 


equations indicate by how much an increase of 0.1 in P(W) contributes to an increase in 











the average MOV for the X team. Based on these equations, it appears that a 0.1 
increase in P(W) increases the average MOV by approximately 2.0 TTTs. 

The R-sq (r-squared) value, which is the correlation coefficient squared is 
shown in Appendix G contains along with the regression equations. For example, at 
Tactical Delay 0, R-sq = 75.2% indicating the regression equation for that delay explains 
75.2% of the variation in the average MOV. The average R-sq value for all four delay 
levels is 68.35%. From these results, it appears P(W) positively influences the average 
MOV. Thus, both of the second research questions and the third as well have been 


answered. 


3. Interactions 

Finally, the fourth research question asks if an interaction exists between 
Tactical Delay and P(W). To determine if interactions exist, two-factor ANOVA was 
used. The same data used in Question 2 was used again so the requirement that the data 
be normal has already been confirmed. The data were manipulated into a single I by J 
matrix to enable the software to analyze it. ANOVA was then run on the data and the 
results are in Appendix H. The area of interest in the ANOVA results is the row labeled 
interaction. Notice the F test statistic there is greater than the F critical value, implying 
that H,,3 is rejected and interactions between delay and P(W) do exist. In following the 
methodology used in the previous two questions, the data was graphed and then 


compared with the results provided by ANOVA. 
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a. Graphical Results 


Figure 2 is also a graph of the interactions. If no interactions existed, 
the four curves would be parallel and wouldn’t intersect. Since they do intersect and the 
results of ANOVA show that interactions exist, it is up to the researcher to determine if 
the interactions are important. If they are not, the factors can be analyzed separately and 
the results of Question 2 are all that need to be considered when analyzing this set of 
data. 

To determine if interactions are important it is necessary to see how 
strong the interactions are. Figure 2 shows that all four curves plot similarly and 
generally have about the same slope. The graph reveals overall a delay level of three 
results in a larger average MOV than a delay of two. The same is true when two is 
compared to one and one to zero. Closer investigation of Figure 2 indicates that eight 
times out of eleven, the average MOV is greatest for delay level three and decreases in 


order down to level zero. 


b. Regression Analysis 
To investigate the graph further, the regression equations generated 
earlier were once again investigated, see Figure 3 or Appendix G. Comparison of the 
slopes will indicate how away from parallel the four curves are. The slopes and y- 


intercepts for all four curves are very similar indicating the best fit curves are very close 


to parallel. 
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Average MOV, = : 
Average MOV, = -9, 69 + 1. 31PCW) 
Average MOV, = -10.5 + 2.01P(W) 
Average MOV, = -8.89 + 1.80P(W) 
Average MOV = -11.6 + 1.90P(W) + 0.573Delay 


Figure 3. Regression Equations 











c. Additional Analysis 
Recall the R-sq value shows the amount of variation between the data sets 
that can be explained by least squares regression. The results here are similar ranging 
from = 55% to = 75%. Additionally, the F values were compared to see if H.,, was 
rejected hy a large amount. The F test statistic is relatively close to the F critical value 
indicating H,,3 was not strongly rejected. 
d. Conclusions 
Based on Figure 2, the regression equations, the similarity between 
correlation coefficients and the small margin between the F test statistic and the F critical 
value, it is the opinion of the author that, although interactions exist between Tactical 
Delay and P(W), they are unimportant and do not effect the average MOV. 
e. Multiple Regression 
Finally, a multiple linear regression was run on the data to analyze the 
combined influence of P(W) and Tactical Delay on average MOV over the entire I by 


J matrix. The regression equation, see the fourth equation in Figure 3 above, indicates 








that P(W) has more than three times the effect on average MOV than Tactical Delay. 
This further corroborates the the graphical analysis that shows the almost complete 
dominance of P(W) over delay. 

This chapter discussed the methodology used to analyze the data and the 
results of the analysis. The final chapter of this thesis will include the conclusions, a 


summary of the experiment and some recommendations for further study in this area. 
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IV. CONCLUSIONS 


The purpose of this thesis was to design, conduct, analyze and report the results 
of a C3 experiment. The T4 Simulation generated data for analysis so the research 
questions in the first chapter could be answered. The remainder of this chapter 


summarizes the thesis results presented earlier. 


A. RESEARCH QUESTION RESULTS 

The data generated and analysis conducted was sufficient to answer the stated 
research questions. The last chapter discussed the data analysis and provided conclusions 
to the research questions posed in the first chapter. Below the questions are answered 
directly, a brief summary of the experiment is provided and the thesis closes with 


recommendations for further studies in this area. 


1. Delay Levels 
It was determined that Tactical Delay of zero was significantly different than 
delays one through nine. Analysis of delays one through nine indicated no significant 
change in average MOV among those delays. Area and Communication Delays showed 
no significant difference in average MOV at any level. Therefore, based on the 
configuration of this experiment, it can be concluded that: 
1. Only Tactical Delay significantly effects average MOV, and 


2. There isa level of delay beyond which average MOV ceases to decline and it 
has been determined to be Tactical Delay = 1. 
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2. P(W) 

This section will answer research questions 2a, 2b and 3. In all cases, 
changing the level of P(W) provides a distinct increase in average MOV for the team 
assigned the greater value of P(W). The average MOV increases linearly with an 
increase in P(W) and at no level does a further increase in P(W) not result in an increase 
in average MOV. It was determined that an increase of 0.1 in P(W) results in an 
increase of approximately two TTTs. These results remained approximately the same 


regardless of the level of Tactical Delay. 


3. Delay and P(W) Interactions 
Finally, it was determined that an interaction existed between Tactical Delay 
and P(W). The interaction was determined to be unimportant and the two factors were 
analyzed separately. Separate analysis was accomplished using single-factor ANOVA 


previously and no further analysis was required. 


B. SUMMARY 

This experiment enabled the researcher to use several statistical tools to analyze 
data generated by the T4 Simulation. The simulation, acting as a surrogate for a C3 
system, provides data which allows the researcher to determine an optimal combination 
of the assigned variables, i.e., an optimal system alignment. In this specific 
configuration the X team player assigned P(W) = 1.0 playing against an O team assigned 
Tactical Delay one or greater will always be more successful than any other 


configuration. 
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This type of experiment lends itself well to systems evaluation. For example, if 


a C3 system were being analyzed to try and determine the optimal setting of system 
parameters, an analysis similar to this one could be conducted and an optimal systems 


alignment could be determined statistically. 


C. RECOMMENDATIONS 

The T4 Simulation is an excellent tool for generating C3 related data to determine 
an optimal T4 configuration. The number of configurations are numerous and the 
spreadsheet generated is full of data to be analyzed. The only recommendations this 
author can suggest is to take advantage of the different configurations available in the T4 
simulation. 

There are several options for a continuation to this thesis. Chapter two points out 
several parameters that were not investigated. Another researcher may attempt to 
replicate the above results using a completely different configuration. After the systems 
evaluation course, C3 students will be very familiar with this simulation and may want 
to propose a configuration that interests them, predict what they think will happen, and 
test their predictions using some or all of the tools presented in this thesis. There are 
vast opportunities for the C3 student who enjoys this type of analysis to pursue this area 


for thesis study. 
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APPENDIX A: CHECKING FOR NORMALITY 


Histogram of Area Delay at level 0, N = 120 
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Histogram of Communication Delay at level 0, N = 120 
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Histogram of Tactical Delay at level 0, N = 120 
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APPENDIX B: SINGLE-FACTOR ANOVA 


Anova:Single-Factor Area Delay 


Summary 

Groups Count Sum Average Variance 
0 120 25 0.208 8.318 
1 120 8 0.067 6.802 
2 120 37 0.308 8.148 
3 120 59 0.492 8.336 
4 120 83 0.692 7.022 
5 120 64 0.533 5.965 
6 120 61 0.508 5.798 
7 120 82 0.683 6.336 
8 120 47 0.392 6.660 
9 120 101 0.842 8.118 
ANOVA 


Source of Variation 

Ss df MS F P-val F crit 
Between Groups 60.3 9 6.7 0.94 0.492 1.89 
Within Groups 8508.8 1190 7.2 


Total 8569.1 1199 








Anova:Single-Factor Comm Delay 


Summary 

Groups Count Sum Average Variance 
0 120 32 0.27 10.6 
1 120 3 0.025 7.991 
2 120 4 0.033 7.427 
3 120 46 0.383 8.205 
4 120 27 0.225 6.058 
5 120 9 0.075 7.818 
6 120 58 0.483 6.806 
7 120 53 0.442 7.912 
8 120 -7 0.058 7.114 
9 120 7 0.058 7.770 
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ANOVA 


Source of Variation 

SS df 
Between Groups 45.8 9 
Within Groups 9242.6 


Total 9288.397 


Anova: Single-Factor 


Summary 
Groups Count Sum 
0 120 -7 
1 120 169 
2 120 201 
3 120 185 
4 120 263 
5 120 244 
6 120 253 
7 120 260 
8 120 176 
9 120 270 
ANOVA 
Source of Variation 

SS 
Between Groups 515.1 
Within Groups = 13454.8 
Total 13969.84 


MS F P-val_—F crit 
3.1 0.655 0.75 1.89 
7.77 
Tactical Delay 

Average Variance 
-0.058 7.971 

1.408 10.328 

1.675 9.986 

1.542 11.679 

2.192 10.711 

2.033 12.520 

2.108 10.770 

2.167 13.165 

1.467 13.327 

2.250 12.609 
df MS F P-value 
9 57.23 $.06 0 
1190 11.3 

1199 
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F crit 
1.89 











APPENDIX C: TUKEY’S RESULTS 


Tukey’s Pairwise Comparisons 

Family error rate = 0.0500 
Individual error rate = 0.00160 
Critical value = 4.47 


Intervals for (column level mean) - (row level mean) 


0 1 2 3 4 5 6 7 8 


2 -3.105 -1.639 
0.361 = 1.105 


3 +-2.972 -1.505 -1.239 
0.228 =1.239. 1.505 


4 -3.622 -2.155 -1.889 -2.022 
0.878 0.589 0.855 0.722 


§ +-3.464 -1.997 -1.730 -1.864 -1.214 
0.720 0.747 1.014 0.880 1.530 


6 -3.539 -2.072 -1.805 -1.939  -1.289 -1.447 
0.795 0.672 0.939 0.805 1.455 1.297 


7 -3.597 -2.130 -1.864 -1.997  -1.347 -1.505 -1.430 
0.853 0.614 0.880 0.747 1.397 1.239 1.314 


8 -2.897 -1.430 -1.164 -1.297 0.647 -0.805 -0.730 -0.672 
0.153. 1.314 1.580 1.447 2.097 1.939 2.014 2.072 


9 -3.680 -2.214 -1.947 -2.080 -1.430 -1.589 -1.514 -1.455  -2.155 
0.936 0.530 0.797 0.664 1.314 1.155 1.230 1.289 0.589 








APPENDIX D: LINEAR REGRESSION 


The regression equation is 


Average MOV = 1.55 + 0.0651 Tactical Delay 


Predictor Coef Stdev t-ratio p 
Constant 1.5456 0.2266 6.82 0.000 
TacDel 0.06514 0.04027 1.62 0. 106 


s = 3.417 R-sq = 0.2% R-sq(adj) = 0.1% 








APPENDIX E: SINGLE-FACTOR ANOVA 


Changing P(W) at Time Delay 0 
Anova: Single-Factor 
Summary 
Groups Count Sum Average Variance 
1.0 120 -1326 11.050 13.964 
0.9 120 -885 -7.375 15.682 
0.8 120 -550 -4.583 15.455 
0.7 120 -334 -2.783 10.659 
0.6 120 -166 -1.383 9.415 
0.5 120 -7 0.058 7.971 
0.4 120 217 1.808 10.223 
0.3 120 360 3.000 7.697 
0.2 120 631 5.258 11.991 
0.1 120 957 7.975 17.319 
0.0 120 1322 11.017 13.126 
ANOVA 
Source of Variation 
SS df MS F P-val F crit 
Between Groups 51809.87 10 5180.987 426.89 0 1.838 


Within Groups = 15886.79 1309 12.13659 


Total 67696.67 1399 
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Changing P(W) at Time Delay 1 
Anova: Single-Factor 


Groups Count Sum Average Variance 


1.0 120 -867 -7.225 36.361 
0.9 120 -164 -6.367 25.125 
0.8 120 475 -3.958 21.250 
0.7 120 -299 -2.492 15.865 
0.6 120 53 -0.442 = 12.249 
0.5 120 169 1.408 10.328 
0.4 120 188 1.567 12.735 
0.3 120 496 4.133 13.696 
0.2 120 702 5.850 15.154 
0.1 120 1004 8.367 16.890 
0.0 120 1412 11.767 = 12.332 


ANOVA 


Source of Variation 

Ss df MS F P-val F crit 
Between Groups 43746.7 10 4374.666 250.653 0 1.838 
Within Groups 22846.1 1309 17.45311 


Total 66592.8 1319 


Changing P(W) at Time Delay 2 
Anova: Single-Factor 


Summary 
Groups Count Sum Average Variance 


1.0 120 = -1332 -11.1 = 16.242 
0.9 120 -109  -5.908 25.042 
0.8 120 -332  --2.767 22.315 
0.7 120 -127  -1.058 12.980 
0.6 120 33. 0.275 «14.789 
0.5 120 219 1.825 12.667 
0.4 120 348 2.9 13.183 
0.3 120 501 4.175 13.809 
0.2 120 861 7.175 17.473 
0.1 120 1137 9.475 18.336 
0.0 120 1467 12.225 9.436 


53 


ANOVA 


Source of Variation 

SS df MS F P-vai F crit 
Between Groups 55188 10 5518.8 344.392 0 1.838 
Within Groups 20976.4 1309 16.02475 
Total 76164.4 1319 


Changing P(W) at Time Delay 3 
Anova: Single-Factor 


Summary 
Groups Count Sum Average Variance 


120 -7.075 32.272 
120 -5.267 30.584 
120 -2.85 21.624 

120 -1.767 16.668 
{20 0.458 15.948 

120 2.142 10.744 
120 2.917 13.859 
120 4.633 16.217 
120 6.775 17.588 
120 8.808 16.190 
120 12.542 9.746 


Source of Variation 

Ss df MS F P-val_ F crit 
Between Groups 43593.62 10 4359.4 238.052 0 1.838 
Within Groups 23971.28 1309 18.3 


Total 67564.91 1319 














APPENDIX F: TUKEY’S RESULTS 


Tukey’s pairwise comparisons for Tactical Delay= 0 


Family error rate = 0.0500 
Individual error rate = 0.00132 


Critical value = 4.55 
Intervals for (column level mean) - (row level mean) 


1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 


0.9 -5.122 
-2.228 


0.8 -7.914 -4.239 
5.020 -1.345 


0.7 -9.714 -6.039 -3.247 
6.820 = -3.145 = -0.353 


0.6 -11.114 -7.439 -4.647 -2.847 
-8.220  -4.545  -1.753 0.047 


0.5 -12.439 -8.764 -5.972 -4.172  -2.772 
-9.545 -5.870 -3.078 -1.278 =: 0.122 


0.4 -14.305 -10.630 -7.839 -6.039 -4.639 -3.314 
“11.411 -7.736 -4.945 -3.145  -1.745 -0.420 


0.3 -15.497 -11.822 -9.030 -7.230 -5.830 -4.505 -2.639 
“12.603 -8.928 -6.136 -4.336 -2.936 -1.611 0.255 


0.2 -17.755  -14.080 -11.289 -9.489 -8.089 -6.764 -4.897 -3.705 
14.861 -11.186 -8.395 -6.595 -5.195  -3.870 -2.003 -0.811 


0.1 


0.1 -20.472 -16.797 -14.005 -12.205 -10.805 -9.480 -7.614 -6.422 -4.164 


“17.578 -13.903 -11L.111 -9.311 -7.911 6.586 -4.720 -3.528 


0.0 -23.514 -19.839 -17.047 -15.247 -13.847 -12.522 -10.655 -9.464 
-20.620 -16.945 -14.153 -12.353 -10.953 -9.628  -7.761 -6.570 


-1.270 


-7.205 
4311 


4.489 
-1.595 





Tukey’s pairwise comparisons for Tactical Delay = 1 


Family error rate = 0.0500 
Individual error rate = 0.00132 


Critical value = 4.55 
Intervals for (column level mean) - (row level mean) 


1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 


0.9 -2.594 
0.877 


0.8 -S.002 -4.144 
-1.531  -0.673 


0.7 6.469 -5.610 -3.202 
-2.998 -2.140 0.269 


0.6 -8.519 -7.660 -5.252 -3.785 
“5.048 -4.190 -1.781 -0.315 


0.5 -10.369 -9.510 -7.102 -5.635 -3.585 
6.898 -6.040 -3.631 -2.165 0.115 


0.4 -10.527 -9.669 -7.260 -5.794 -3.744 -1.894 
-7.056 6.198 -3.790 -2.323 -0.273 1.577 


0.3 -13.094 -12.235 -9.827 -8.360 -6.310 -4.460 -4.302 
-9.623 -8.765 -6.356 -4.890 -2.840 -0.990 -0.831 


0.2 -14.810 -13.952 -11.544 -10.077 -8.027 -6.177 -6.019 -3.452 
11.340 -10.481 -8.073 -6.606 -4.556 -2.706 -2.548 0.019 


0.1 -17.327 -16.469 -14.060 -12.594 -10.544 -8.694 -8.535 -5.969 -4.252 
-13.856 -12.998 -10.590 -9.123 -7.073 -5.223 -5.065 -2.498 -0.781 


0.0 -20.727 -19.869 -17.460 -15.994 -13.944 -12.094 -11.935 -9.369 -7.652 -5.135 
-17.256 -16.398 -13.990 -12.523 -10.473 -8.623 -8.465 -5.898 -4.181 -1.665 


Tukey’s pairwise comparisons for Tactical Delay = 2 


Family error rate = 0.0500 
Individual error rate = 0.00132 


Critical value = 4.55 


Intervals for (column level mean) - (row level mean) 


0.9 


0.8 


0.7 


0.6 


0.5 


0.4 


0.3 


0.2 


0.1 


0.0 


1.0 


6.854 
-3.529 


-9.996 
6.671 


-11.704 
-8.379 


-13.038 
-9.712 


-14.588 
11.262 


-15.663 
-12.337 


-16.938 
-13.612 


-19.938 
-16.612 


-22.238 
18.912 


-24.988 
-21.662 


0.9 


~4.804 
-1.479 


6.513 
-3.187 


-7.846 
4.521 


-9.396 
-6.071 


-10.471 
-7.146 


-11.746 
-8.421 


-14.746 
11.421 


-17.046 
-13.721 


-19.796 
-16.471 


0.8 


-3.371 
-0.046 


~4.704 
-1.379 


-6.254 
-2.929 


-7.329 
-4.004 


-8.604 
-5.279 


11.604 
-8.279 


-13.904 
-10.579 


-16.654 
-13.329 


0.7 


-2.996 
0.329 


4.546 
-1.221 


-5.621 
-2.296 


6.896 
-3.571 


-9.896 
6.571 


12.196 
8.871 


-14.946 
11.621 


0.6 


-3.213 
0.113 


4.288 
-0.962 


-5.563 
-2.237 


-8.563 
-5.237 


-10.863 
7.537 


13.613 
-10.287 


57 


0.5 


-2.738 
0.588 


-4.013 
-0.687 


7.013 
-3.687 


9.313 
-5.987 


-12.063 
-8.737 





0.4 0.3 
-2.938 
0.388 
-5.938 -4.663 
-2.612 -1.337 
-8.238 -6.963 
4.912 -3.673 


0.2 


-3.963 
-0.637 


0.1 


“10.988 -9.713 -6.713 ~4.413 


-7.662 -6.387 


-3.387 


~1.087 





Tukey's pairwise comparisons for Tactical Delay = 3 


Family error rate = 0.0500 
Individual error rate = 0.00132 


Critical value = 4.55 
Intervals for (column level mean) - (row level mean) 


1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 


0.9 -3.586 
-0.031 


0.8 6.002 ~-4.194 
-2.448 = -0.639 


0.7 -7.086 -5.277  -2.861 
-3.531 -1.723 0.694 


0.6 -9.311 -7.502 -5.086 -4.002 
-5.756 -3.948 -1.531 0.448 


0.5 -10.994 -9.186 -6.769 -5.686 -3.461 
-7.439 -5.631 -3.214 -2.131 0.094 


0.4 -11.769 -9.961 -7.544 -6.461 -4.236 -2.552 
“8.214 -6.406 -3.989 -2.906 -0.681 1.002 


0.3 -13.486 -11.677 -9.261 -8.177 -5.952 -4.269 -3.494 
9.931 -8.123 -5.706 -4.623 -2.398 -0.714 0.061 


0.2 -15.627 -13.819 -11.402 -10.319 -8.094 -6.41] -5.636 -3.919 
-12.073 -10.264 -7.848 -6.764 -4.539 -2.856 -2.081 ~-0.364 


0.1 -17.661 -15.852 -13.436 -12.352 -10.127 -8.444 -7.669 -5.952 -3.811 
“14.106 -12.298 -9.881  -8.798 -6.573  -4.889 -4.114 -2.398 0.256 


0.0 -21.394 -19.586 -17.169 -16.086 -13.861 -12.177 -11.402 -9.686 -7.544 -5.511 
17.839 -16.031 -13.614 -12.531 -10.306 -8.623 -7.848 6.131 -3.989 -1.956 








APPENDIX G: LINEAR REGRESSION 


The regression equation at Tactical Delay 0 is 
Average MOV = - 11.6 + 1.96 P(W) 


Predictor Coef Stdev _t-ratio p 
Constant -11.6168 0.2107 -55.13 0.000 
C46 1.96379 0.03107 63.21 0.000 


8 = 3.569 R-sq = 75.2%  R-sq(adj) = 75.2% 
The regression equation at Tactical Delay i is 
Average MOV= - 9.69 + 1.81 P(W) 

Predictor Coef Stdev _t-ratio p 
Constant -9.6852 0.2497 -38.79 0.000 
C48 1.80523 0.03681 49.04 0.000 
s = 4.229 R-sq = 64.6%  R-sq(adj) = 64.6% 
The regression equation at Tactical Delay 2 is 
Average MOV = - 10.5 + 2.01P(W) 

Predictor Coef Stdev _t-ratio P 
Constant -10.4935 0.2458 ~42.69 0.000 
C50 2.00977 0.03624 55.46 0.000 
s = 4.163 R-sq = 70.0%  R-sq(adj) = 70.0% 
The regression equation at Tactical Delay 3 is 
Average MOV = - 8.89 + 1.80P(W) 

Predictor Coef Stdev _t-ratio P 
Constant -8.8903 0.2549 -34.88 0.000 
C52 1.80470 0.03758 48.02 0.000 


$= 4.318  R-sq= 63.6% R-sq(adj) = 63.6% 
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APPENDIX H: TWO-FACTOR ANOVA 


Anova: Two-Factor With Replication 
Summary 


DelayO Delay 1 Delay 2 Delay 3 Total 


P(W)=1.0 
Count 120 120 120 120 480 
Sum -1326 -867 -1332  -849 -4374 


Average “11.05 -7.23 -M1.1 0 -7.075  -36.45 
Variance 13.96 36.36 16.24 32.27 98.84 


P(W)=0.9 
Count 126 120 120 120 480 
Sum “885 -764 -709 -632 -2990 


Average -7.375 6.37 -5.91  -5.23 -24.92 
Variance 15.68 25.12 25.04 30.58 96.43 


P(W) =0.8 
Count 120 120 120 120 480 
Sum -550 ~475 -332 -342 -1699 


Average -4.58 -3.96 -2.77 -2.85 -14.16 
Variance 15.46 21.25 22.31 21.624 80.64 


P(W)=0.7 
Count 120 120 120 120 480 
Sum -334 -299 -127 -212 -972 


Average -2.78 -2.49  -1.0% -1.77 -8.1 
Variance 10.66 15.87 12.98 16.67 56.17 


P(W)=0.6 
Count 120 120 120 120 480 
Sum -166 -53 33 55 -131 


Average -1.38 -0.44 0.275 0.46 -1.09 
Variance 9.41 12.25 14.79 = 15.95 52.40 











P(W)=0.5 


Count 120 120 120 120 480 
Sum -7 169 21.9 257 638 
Average -0.06 1.41 1.83 2.14 5.32 
Variance 7.97 10.33 12.67 10.74 41.71 


P(W)=0.4 


Count 120 120 120 120 480 
Sum 217 188 348 350 1103 
Average 1.81 1.57 2.9 2.92 9.19 
Variance 10.22 12.74 13.18 13.86 50.00 


P(W)=0.3 


Count 120 120 120 120 480 
Sum 360 496 501 556 1913 
Average 3 4.13 4.18 4.63 15.94 
Variance 7.70 13.70 13.81 16.22 51.42 


P(W)=0.2 


Count 120 120 120 120 480 
Sum 631 702 861 813 3007 
Average 5.26 5.85 7.18 6.78 25.06 
Variance 11.99 15.153 17.47 17.59 62.21 


P(W)=0.1 


Count 120 120 120 120 480 
Sum 957 1004 1137 1057 4355 
Average 7.98 8.37 9.48 8.81 34.625 
Variance 17.32 16.89 18.34 16.19 68.73 


P(W) =0.0 


Count 120 120 120 120 480 
Sum 1322 1412 1467 1505 5706 
Average 11.02 11.77 12.23 12.54 47.55 
Variance 13.13 12.33 9.44 9.75 44.64 





Total 


Count 1320 1320 1320 1320 
Sum 219 1513 2066 2558 
Average 1.83 12.61 17,22 21.32 
Variance 133.50 191.98 176.27 201.44 


ANOVA 

Source of Variation SS = df MS F P-val 
Sample 192490.92 10 19249.09 1204.44 

Column 2309.97 3 769.99 48.18 
Interaction 1847.23 30 61.57 3.85 

Within 83680.60 5236 15.98 

Total 280328.72 5279 
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APPENDIX I: MULTIPLE LINEAR REGRESSION 


The regression equation is 
MOV = - 11.6 + 1.90 P(W) + 0.573 Delay 


Predictor Coef Stdev t-ratio p 
Constant -11.6052 0.1745 -66.51 0.000 
P(W) 1.89587 0.01781 106.45 0.000 
Delay 0.57348 0.05037 11.38 0.000 


s= 4.092 R-sq= 68.5% R-sq(adj) = 68.5% 
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